Author Topic: File decoder slowing down?  (Read 942 times)

ddddd13

  • Full Member
  • ***
  • Posts: 118
    • View Profile
File decoder slowing down?
« on: July 13, 2019, 10:37:24 AM »
The following is code to open the oldest file, decode it, save it, and move the original file to a new directory every 4 seconds. It works great, except!

When decoding a large amount of files it slows down. Dramatically! I added the EpicorData() clear to clean out the List of String, and that helped a lot. I am also using other lists of string but do not want to clean them out every time.

I noticed when I went to look at the directory with file explore it would suddenly decode every 4 seconds again. This was happening because the form Visible routine was being run from the beginning cleaning things out, I think!

So I decided to add another form that I will call every 20th file read. That would close the main form and open form3.
When form 3 was active or visible it would close form 3 and reopen the Main form. I found a routine on the forum

If Myform isnot nothing then
Myform=new form3
end if
Myform.Show

My question is. Don't I have to close the form I am on? If the above code is in the Main form then the following should be in the form3 visible routine?

If Myform isnot nothing then
Myform=Main form
end if
Myform.show

Thanks,
Dave
Imports System.Net
Imports System.Net.Security

Imports System.IO

Public Class MainForm

    Dim Machine As String
    'Dim Employee As String
    Dim file As String
    Dim EMPLOYEES As New List(Of String)
    Dim MACHINENUM As New List(Of String)
    Dim parameter As New List(Of String)
    Dim EpicorData As New List(Of String)
    Dim LenOfDir As New List(Of Integer)
    Dim FtpAddress As String
    Dim FtpUserName As String
    Dim FtpPassword As String
    Dim FtpFileName As String



    '*******************************************************************************
    '* Stop polling when the form is not visible in order to reduce communications
    '* Copy this section of code to every new form created
    '*******************************************************************************
    Private NotFirstShow As Boolean

    Private Sub Form_VisibleChanged(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.VisibleChanged
        '* Do not start comms on first show in case it was set to disable in design mode
        If NotFirstShow Then
            AdvancedHMIDrivers.Utilities.StopComsOnHidden(components, Me)
        Else
            NotFirstShow = True
        End If
    End Sub

    '***************************************************************
    '* .NET does not close hidden forms, so do it here
    '* to make sure forms are disposed and drivers close
    '***************************************************************
    Private Sub MainForm_FormClosing(sender As Object, e As FormClosingEventArgs) Handles Me.FormClosing
        Dim index As Integer
        While index < My.Application.OpenForms.Count
            If My.Application.OpenForms(index) IsNot Me Then
                My.Application.OpenForms(index).Close()
            End If
            index += 1
        End While
    End Sub
    Private Sub MainForm_Activated(sender As Object, e As EventArgs) Handles MyBase.Activated
        'setup length of file variables
        Debug.WriteLine("Activated")
        For i = 0 To 31
            LenOfDir.Insert(i, 0)
        Next i
        'read parameters to find ftp address ect.
        Using sr As New System.IO.StreamReader("C:/fILES/EpicParm.txt")
            While Not sr.EndOfStream
                parameter.Add(sr.ReadLine())
            End While
        End Using
        TextBox1.Text = parameter(1)
        'read EMPLOYEES to MATCH NUMBER TO NAME
        Using sr As New System.IO.StreamReader("C:/files/EMPLOYEEs.TXT")
            While Not sr.EndOfStream
                EMPLOYEES.Add(sr.ReadLine())
            End While
        End Using
        'read MACHINE to MATCH NUMBER TO NAME
        Using sr As New System.IO.StreamReader("C:/files/MACHINE.TXT")
            While Not sr.EndOfStream
                MACHINENUM.Add(sr.ReadLine())
            End While
        End Using

        RichTextBox1.Text = MACHINENUM.Count

        FtpAddress = parameter(0)
        FtpUserName = parameter(1)
        FtpPassword = parameter(2)
        ' RichTextBox1.Text = FtpAddress & Chr(13) & FtpUserName & Chr(13) & FtpPassword & Chr(13) & parameter(3) & Chr(13) & parameter(4) & Chr(13) & parameter(5)
    End Sub


    Private Sub MainForm_Load(sender As Object, e As EventArgs) Handles MyBase.Load

        ' Timer1.Enabled = True
        Timer2.Enabled = True
    End Sub

    Private Sub Timer1_Tick(sender As Object, e As EventArgs) Handles Timer1.Tick

    End Sub

    Private Sub Timer2_Tick(sender As Object, e As EventArgs) Handles Timer2.Tick
        Dim LineCount As Integer = 0
        Dim fileCount As Integer = IO.Directory.GetFiles("C:\Epicor_Programs", "*.*").Length
        'if a file exists decode it
        If fileCount > 0 Then
            EpicorData.Clear()
            'get the directory and display in text box 4
            Dim files() As String = IO.Directory.GetFiles("c:\Epicor_Programs")
            RichTextBox4.Text = files(0)

            'read the oldest FILE TO FIND MACHINE INFO FROM EPICOR
            Using sr As New System.IO.StreamReader(files(0))
                TextBox3.Text = (files(0))
                While Not sr.EndOfStream
                    EpicorData.Add(sr.ReadLine())
                    TextBox3.Text = TextBox3.Text + (LineCount.ToString) & (vbCrLf)
                End While
            End Using

            'convert epicor data into single strings
            For i = 0 To EpicorData.Count - 1
                RichTextBox6.Text = (EpicorData(i))
                RichTextBox5.Text = RichTextBox6.Text.Replace(",", vbCrLf)

                'find employee number and match it to name
                'Then reinsert it into textbox 5 for saving to file
                For j = 0 To (EMPLOYEES.Count - 1)
                    RichTextBox1.Text = RichTextBox1.Text + EMPLOYEES(j).Substring(0, 4) & (vbCrLf)
                    If EMPLOYEES(j).Substring(0, 4) = RichTextBox5.Lines(0) Then
                        Dim lines() As String = Me.RichTextBox5.Lines
                        lines(0) = EMPLOYEES(j)
                        Me.RichTextBox5.Lines = lines
                    End If
                Next j

                'find MACHINE NAME and match it to number
                'Then reinsert it into textbox 5 for saving to file
                Dim Machines2() As String
                For k = 0 To (MACHINENUM.Count - 1)


                    Machines2 = MACHINENUM(k).Split(",")
                    RichTextBox2.Text = RichTextBox2.Text + Machines2(0) & (vbCrLf)
                    If Machines2(0) = RichTextBox5.Lines(1) Then
                        Dim lines() As String = Me.RichTextBox5.Lines
                        lines(1) = MACHINENUM(k)
                        Me.RichTextBox5.Lines = lines
                    End If
                Next k

                ' Write decoded data to file with machine number for name
                Machine = RichTextBox5.Lines(1)
                TextBox3.Text = Machine
                'Check if data exists in machine string
                If Machine.Length > 2 Then
                    Machine = Machine.Substring(Machine.Length - 3)
                End If
                'If it doesnt have a machine number asign it noname
                If Machine.Length < 1 Then
                    Machine = "noname"
                End If
                Dim strWrite As New StreamWriter("C:\MachineData\" & Machine & ".txt")

                For Each line As String In RichTextBox5.Lines
                    strWrite.WriteLine(line)
                Next

                strWrite.Close()
            Next i

            'GET THE FILENAME AND COPY IT TO EPICOR PROGRAM OLD DIRECTORY
            Dim file2 As String
            file = RichTextBox4.Lines(0)
            file2 = file.Substring(file.Length - 34)
            TextBox1.Text = file
            My.Computer.FileSystem.MoveFile(file, "C:\Epicor_Programs_Old\" & file2)
         
        End If

    End Sub


End Class

Archie

  • Administrator
  • Hero Member
  • *****
  • Posts: 5322
    • View Profile
    • AdvancedHMI
Re: File decoder slowing down?
« Reply #1 on: July 13, 2019, 11:52:23 AM »
I haven't looked at this in detail yet, but the one thing I would add is in the TimerTick event to stop the timer at the beginning and restart at the end.

Code: [Select]
Private Sub Timer2_Tick(sender As Object, e As EventArgs) Handles Timer2.Tick
      Timer2.Enabled=False;
      Dim LineCount As Integer = 0
.
.
.
.
    Timer2.Enabled=True;
End Sub

This will guarantee the timer does not try to re-fire before the processing is done.

ddddd13

  • Full Member
  • ***
  • Posts: 118
    • View Profile
Re: File decoder slowing down?
« Reply #2 on: July 13, 2019, 01:20:57 PM »
Archie;

I added your code and dropped the timer to half a second. It will write a file about once every second in the beginning. When it gets to 50 files it is once every 4 seconds. Stop debug and restart it, again it goes about once every second slowing down to about one every 4 seconds after 50 files.

I look through two lists, EMPLOYEES, and MACHINENUM. This has to somehow increase the memory used by the program and slow it down. If I can restart the entire routine every 20th file I will be fine. I have tried creating a new form and doing a Form3.show then in form3 doing a MainForm.Show but that doesn't work.


???
Dave

ddddd13

  • Full Member
  • ***
  • Posts: 118
    • View Profile
Re: File decoder slowing down?
« Reply #3 on: July 13, 2019, 02:07:17 PM »
Archie;

I FOUND IT!!!!! I was appending to the RichTextBox. This made it huge after a couple hundred files and slowed it down. Runs perfect now.

Thanks,
Dave

ddddd13

  • Full Member
  • ***
  • Posts: 118
    • View Profile
Re: File decoder slowing down?
« Reply #4 on: July 13, 2019, 02:18:41 PM »
over 200 files in less than 20 seconds!!! Who Hoo!!!