Monday, March 7, 2016

Weighting logistic loss for imbalanced dataset in Spark

How to apply corrective weight for the training of a logistic regression with imbalanced dataset using Apache Spark MLlib?
Some applications such as spam or online targeting have an imbalanced dataset. The number of observations associated to one label is very small (minority class) compared to the number of observations associated to the other labels.

Overview
Let's consider the case of intrusion detection system that identifies an cyber attack through a Virtual Private Network (VPN) in an entreprise. The objective is to classify every session based on one or several TCP connections as potentially harmful intrusion or normal transaction knowing that a very small fraction of attempt to connect and authenticate are actual deliberate attacks (less than 1 for each million of connections).
There are few points to consider:
  • The number of reported attack is very likely very small (< 10). Therefore, the very small set of labeled security breach is plagued with high variance
  • A very large and expensive labeling operation is required, if even available, to generate a fairly stable negative class (security breach)
  • The predictor can be extremely accurate (as measured by F1-score, Area under the ROC curve or PR curve) by always classifying any new attempt to connect to the VPN as harmless. In the case of 1 reported attack per 1 million VPN session, the prediction would be accurate 99.999% of the time.

There are several approaches to address the problem of imbalanced training and validation sets. These list of the most common known techniques onclude
  • Sub-sampling the majority class (i.e. harmless VPN sessions): It reduces the number of labels for the normal sessions while preserving the labeled reported attacks.
  • Over-sampling the minority class: This technique generates synthetic sampling using bootstrapped samples based on k Nearest Neighbors algorithm
  • Application of weights differential to the logistic loss used in the training of the model
This post focuses on the third option or re-balancing (weighting) of the logistic loss. The implementation of this technique is specific to Apache Spark and more specifically the ML implementation of the Logistic Regression (not MLlib)that leverages data frames.
It is assumed the reader is somewhat familiar the Apache Spark, data frame and its machine learning module MLlib.

Weighting the logistic loss
Let's consider the logistic loss commonly used in training a binary model f with feature x and label y: \[logLoss = - \sum_{i=0}^{n-1}[y_i .log(f(x_i)) + (1 - y_i) .log(1 - f(x_i))]\] The first component of the loss function is related to the minority observations (security breach through a VPN: label = 1) while the a second component represents the loss related to the majority observation (harmless VPN sessions: label = 0)
Let's consider the ratio, r of number of observations related to the majority label over the total number of observations: \[r = \frac{ i: (y_i = 1)}{i : y_i}\] The logistic loss can be then rebalanced as \[logloss = -\sum_{i=0}^{n-1} [r.y_i.log(f(x_i)) + (1-r).(1-y_i).log(f(x_i))]\] The next step is to implement the weighting of the binomial logistic regression classifier using Apache Spark.

Weighting the logistic loss
Apache Spark is a open source framework for in-memory processing of large datasets (think as Hadoop on steroids). Apache Spark framework contains a machine learning module known as MLlib. The objective is to modify/override the train method of the LogisticRegression.
One simple option is to sub-class LogisticRegression class in the mllib/ml package. However the logistic loss is actually computed in the private class LogisticAggregatorb which cannot be overridden.
However if you browse through the ml.classification.LogisticRegression.train Scala code, you notice that the class Instance that encapsulates labeled data for the computation of the loss and the gradient of loss has three parameters
  • label: Double
  • feature: linalg.Vector
  • weight: Double

The plan is to use this 3rd parameter weight as the balancing weight ratio r. This is simply accomplished by adding an extra column weightCol to the input data frame dataset and define its value as
  • balancingRatio r for label = 1
  • 1 - balancingRatio (1 - r) for label = 0
A simple implementation of the Weighted Logistic Regression using Apache Spark MLlib implementation of the Logistic Regression :

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
final val BalancingWeightColumnName: String = "weightCol"
final val BalanceRatioBounds = (0.001, 0.999)
final val ImportanceWeightNormalization = 2.0

class WeightedLogisticRegression(balanceRatio: Double = 0.5) 
   extends LogisticRegression(UUID.randomUUID.toString) 
      with Logging {

  this.setWeightCol(BalancingWeightColumnName)

  private[this] val balancingRatio = 
    if (balanceRatio < BalanceRatioBounds._1) 
       BalanceRatioBounds._1
    else if (balanceRatio > BalanceRatioBounds._2) 
       BalanceRatioBounds._2
    else
       balanceRatio

  override protected def train(dataset: DataFrame): 
      LogisticRegressionModel = {
    val newDataset = dataset.withColumn("weightCol", lit(0.0))

    val weightedDataset: RDD[Row] = dataset.map(row => {
      val w = if (row(0) == 1.0) balancingRatio 
         else 1.0 - balancingRatio
      Row(row.getAs[Double](0), row.getAs[Vector](1), 
           ImportanceWeightNormalization * w)
    })

    val weightedDataFrame = dataset.sqlContext
        .createDataFrame(weightedDataset, newDataset.schema)
    super.train(weightedDataFrame)
  }
}

Notes
  • The balancing ratio has to be normalized by a factor ImportanceWeightNormalization = 2: The factor is require to produce a weight of 1 for both the majority and minority classes from a fully balanced ratio of 0.5.
  • The actual balancing ratio needs to be constrained within an acceptable range BalanceRatioBounds to prevent for the minority class to have an outsize influence of the weights of the logistic regression model. In the extreme case, there may not even be a single observation in the minority class (security breach through the VPN). These minimum and maximum values are highly dependent on the type of application.

Here is an example of application of the WeightedLogisticRegression on a training data frame labeledData. The number of data points (or observations) associated to label = 1 is extracted through a simple filter.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
val numPositives = trainingDF.filter("label == 1.0").count
val datasetSize = labeledData.count
val balanceRatio = (datasetSize- numPositives).toDouble/datasetSize

val lr = new WeightedLogisticRegression(balanceRatio)
          .setMaxIter(20)
          .setElasticNetParam(0)
          .setFitIntercept(true)
          .setTol(1E-6)
          .setRegParam(1E-2)

val model = lr.fit(trainingDF)

One simple validation of this implementation of the weighted logistic regression on Apache Spark is to verify that the weights of the logistic regression model generated with the WeightedLogisticRegression with a balancing ratio of 0.5 are very similar to the weights generated by the logistic regression model of the Apache Spark MLlib (ml.classification.LogisticRegressionModel)

Note: This implementation relies on Apache Spark 1.6.0. The latest implementation of the Logistic Regression in beta release of Apache Spark 2.0 does not allow to override the LogisticRegression outside the scope of Apache Spark MLlib.

References

104 comments:

  1. The expansion of internet and other business intelligence leads to large volume of data. Industries are looking for talented professionals to maintain and process huge volume of data with latest tools available in the market. Taking Hadoop Training in Chennai | Big Data Training in Chennai will ensure better career prospects for talented professionals.

    ReplyDelete
  2. Very interesting content which helps me to get the in depth knowledge about the technology. To know more details about the course visit this website.
    hadoop training in chennai | Big Data Training in Chennai

    ReplyDelete
  3. An excellent example of a good article!
    I would recommend using a cloud storage server, the resource protects all your personal data and you are not afraid about virus attacks virtual data room pricing

    ReplyDelete
  4. The result being that the auction could not be as competitive as it had the potential to be, and with low revenues from auctions of surplus goods for the government. Liteblue login

    ReplyDelete
  5. your blog contain very useful information and good points were stated in the blog which are very helpful one, thank you for sharing. Software Testing Training Institute in Chennai | Selenium Training Institute in Chennai | ISTQB Training Institute in Chennai

    ReplyDelete
  6. LiteBlue Official Direct access is provided to LiteBlue Login Online USPS (Liteblue Sign In Gov) where we are going to brief you all on the procedure to log into the main portal of the USPS services.
    usps lite blue

    ReplyDelete
  7. Interesting post! This is really helpful for me. I like it! Thanks for sharing..Data Mining Projects Center in Chennai | Data Mining Projects Center in Velachery

    ReplyDelete
  8. Needed to compose you a very little word to thank you yet again regarding the nice suggestions you’ve contributed here.

    https://www.besanttechnologies.com/training-courses/java-training

    ReplyDelete
  9. It is a one of the great discussion which is very essential for me as well. I must follow the handy discussion and sure that the content will be very useful to me as well. Keep it up. 
    Six Sigma Certification Training in Chennai | Linux Certification Training in Chennai | Microsoft Certification Training in Chennai

    ReplyDelete
  10. Thanks for your marvelous posting on weighting logistics! It is very useful and good. Keep updating...
    Final Year

    Project Center in Chennai
     | IEEE Project Center in Chennai |  Diploma Project Center in Chennai

    ReplyDelete
  11. Really an amazing article!!..I gain lot of information from your blog.keep sharing.

    Electrical Project Center in Chennai | Best Electrical projects in Velachery

    ReplyDelete
  12. Wow Very Nice !! Article providing here very nice information am getting from your website... very nice information am getting from your website.. Again Very Nice
    Best Graphics Designing Training Institute in Chennai | No.1 Graphics Designing Training Institute in Velachery

    ReplyDelete
  13. Good to learn something new about web design from this blog. Thanks for sharing such worthy article.
    Best CorelDraw Training Institute in Chennai | No.1 Photoshop Training Institute in Chennai

    ReplyDelete
  14. Great Post, I believe there are many more pleasurable opportunities ahead for individuals that looked at your site.

    ReplyDelete
  15. I simply wanted to write down a quick word to say thanks to you for those wonderful tips and hints you are showing on this site.

    IBM BPM Online Training in Chennai
    IBM BPM Online Training

    ReplyDelete
  16. This is really an awesome article. Thank you for sharing this.It is worth reading for everyone. Visit us:
    AWS Certification Training in Chennai | AWS Training Institute in Pallikaranai

    ReplyDelete
  17. Good and more informative post... thanks for sharing your ideas and views... keep rocks and updating.........

    PCB Designing Training in Chennai | PCB Training Institute in Chennai | PCB Training in Velachery

    ReplyDelete
  18. Nice Post! It is really interesting to read from the beginning and Keep up the good work and continue sharing like this.

    Linux Certification Training in chennai | Linux Training Institute in Chennai | Linux Exam Center in Chennai

    ReplyDelete
  19. Good Post! Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge.

    MatLab Training Institute in Chennai | MatLab Training in Velachery | MatLab Courses in Medavakkam

    ReplyDelete
  20. Your Blog is really awesome with useful information and informative article.Thanks for sharing such a wonderful and excellent post with us.keep updating such a amazing post..
    ISTQB Certification Training Center in Chennai | ISTQB Certification Exams in Velachery | ISTQB Certification Training in Velachery | ISTQB Certification Exams in Madipakkam

    ReplyDelete
  21. Thanks for sharing your wonderful and very useful information.keep updating such a impressive and attractive blog with interesting content.

    Java Training in Chennai | Java Training in Velachery | Java Training Center in Medvakkam | Java Training in Pallikaranai | Java Courses in Chennai | Java Online Training in Guindy

    ReplyDelete
  22. Good article!!! Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge as updated one, keep blogging…

    Best UI Path Training Institute in Chennai | Best UI Path Training in Velachery | Best UI Path Certification Training in Pallikaranai

    ReplyDelete
  23. Good article!!! Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge as updated one, keep blogging…

    Best UI Path Training Institute in Chennai | Best UI Path Training in Velachery | Best UI Path Certification Training in Pallikaranai

    ReplyDelete
  24. Thanks for sharing your informative blog with useful information,its really very interesting and happy to read your article.keep updating such a wonderful post with us..

    Embedded System Training Institute in Chennai | Embedded Training in Velachery | Embedded System Training in Pallikaranai | Embedded Training in Madipakkam

    ReplyDelete
  25. your Blog is really useful for me,its really very interesting and informative information post.keep updating such an amazing post.
    Linux Certification Training in Chennai | Linux Training Institute in Chennai | Linux Training Center in Adayar | Best Linux Training in Pallikaranai

    ReplyDelete

  26. Your Blog is really amazing; it’s really very informative content and useful information. Thanks for sharing your wonderful blog. Keep updating such a creative knowledge.

    Best Java Training Institute in Chenna | Java Training in Velachery | Best Java Courses in Medavakkam | Java Training Center in Pallikaranai

    ReplyDelete
  27. Your Blog is really amazing,its really useful for me and informative content with helpful information.keep updating such a wonderful post..

    Tally ERP9 Training in Chennai | Best Tally Training in Chennai | Tally Training Center in Pallikaranai | No.1 Tally ERP9 with GST Courses in Velachery

    ReplyDelete
  28. Very informative blog. Helps to gain knowledge about new concepts and techniques.Thanks a lot for sharing this wonderful blog.keep updating such a excellent post..

    Best PCB Design Training in Chennai | No.1 PCB Design Course in Velachery | Best PCB Training in Chennai

    ReplyDelete
  29. Nice blog. Thank you for sharing. The information you shared is very effective for learners I have got some important suggestions from it..
    Python Certification Training Institute in Chennai | Python Training in Chennai | Python Exam Center in Velachery | Python Training in Velachery

    ReplyDelete
  30. It is amazing and wonderful to visit your site.Thanks for sharing this information,this is useful to me...

    AWS Certification Training Institute in Chennai | AWS Training Center in Chennai | AWS Certification Training in Velachery

    ReplyDelete
  31. Thanks for sharing this information,this is helpful to me a lot...It is amazing and wonderful to visit your site.

    Web Designing Training Institute in Chennai | Web designing Training in Velachery | Web Design Training Center in Velachery

    ReplyDelete
  32. Pretty article! I found some useful information in your blog, it was awesome to read, thanks for sharing this great content to my vision, keep sharing.
    Best MicroSoft Azure Training Institute in Chennai | Azure Training in Pallikaranai | Best Azure Certification Training in Pallikaranai | Best Azure Training Center in Chennai

    ReplyDelete
  33. This is a great inspiring article.I am pretty much pleased with your good work. You put really very helpful information. Keep it up. Keep blogging. Looking to reading your next post..

    Best ISTQB Certification Training in Chennai | No.1 ISTQB Certification Training Center in Chennai | ISTQB Certification Exam Center in Velachery | ISTQB Certification Training in Saidapet

    ReplyDelete
  34. Thank you so much for posting this. I really appreciate your work. Keep it up. Great work! I am really interested to continue reading your blog. You have shared valid info. Waiting for more updates from you.
    AWS Certification Training in Chennai | Best AWS Training Institute in Chennai | No.1 AWS Certification Training in Nanganallur | AWS Training in Velachery

    ReplyDelete
  35. It is awesome and wonderful to visit your site.Thanks for sharing this information,this is useful to me...
    PCB Design Training Institute in Chennai | PCB Designing Training in Velachery | PCB Design in Pallikaranai | PCB Course in Chennai

    ReplyDelete
  36. Good article! Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge as updated one, keep blogging…
    Python Certification Training in Chennai | Python Training Institute in Velachery | Python Certification Exams in Chennai | Python Exam Center in Chennai

    ReplyDelete
  37. Wow...What an excellent informative blog, really helpful. Thank you so much for sharing such a wonderful article with us.keep updating..

    MatLab Training Institute in Chennai | MatLab Training in Velachery | Matlab Training Center in Chennai | MatLab Courses in Pallikaranai | MatLab Training with Placement in Chennai

    ReplyDelete
  38. I feel really happy to see your blog and look forward to many more entertaining times reading here. Thanks once more for all the details.

    Java Training institute in Chennai | Java Training Center in Chennai | Java Courses in Velachery

    ReplyDelete
  39. It is awesome and wonderful to visit your site.Thanks for sharing your ideas and views... keep rocks and updating........
    Tally ERP9 Training Institute in Chennai | Tally Training Center in Velachery | Tally Training Center in Taramani

    ReplyDelete
  40. I have read your blog its very attractive and impressive. I like it your blog.Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge.

    VMware Certification Training in Chennai | VMware Certification Exam Center in Chennai | VMware Exams Center in Taramani | VMware Certification Exams in Chennai

    ReplyDelete
  41. Wow...What an excellent informative blog, really helpful. Thank you so much for sharing such a wonderful post with us.keep updating..
    AWS Certifications in Chennai | AWS Exam Centers in Chennai | AWS Certification Exams in Velachery | AWS Exams in Velachery | AWS Online Exam Center in Velachery

    ReplyDelete
  42. Very informative and interesting blog, it was so good to read and useful to improve my knowledge as updated one,keep updating..This Concepts is very nice Thanks for sharing.
    ISTQB Certification in Chennai | ISTQB Exam Centers in Chennai | Best ISTQB Exams in Velachery

    ReplyDelete
  43. It is really very awesome and wonderful to visit your site.Thanks for sharing your informative blog with us.keep updating such a wonderful post..
    MicroSoft Azure Certification in Chennai | Azure Exam Centers in Velachery | Azure Exam Centers in Madipakkam

    ReplyDelete
  44. Your Blog is really amazing with useful and helpful content for us.Thanks for sharing.keep updating more information.
    Embedded System Training Institute in Chennai | Embedded Training in Velachery | Embedded System Training in Guindy

    ReplyDelete
  45. Impressive blog with lovely information. really very useful article for us thanks for sharing such a wonderful blog...
    Java Training institute in Chennai | Java Certification Training Center in Velachery | Java Training in Pallikaranai

    ReplyDelete
  46. Nice Post! It is really interesting to read from the beginning and Keep up the good work and continue sharing like this.
    Linux Training Institute in Chennai | Linux Certification Training in Velachery

    ReplyDelete
  47. Pretty article! I found some useful information in your blog, it was awesome to read, thanks for sharing this great content to my vision...
    CCNA Training Institute in Chennai | CCNA Training Center in Velachery

    ReplyDelete
  48. Excellent Post! Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge.
    Java Training Institute in Chennai | Java Certification Training in Velachery

    ReplyDelete
  49. Thanks for sharing your wonderful information..Its really impressive and informative content..
    AWS Certification Training in Chennai | AWS Training Institute in Chennai | AWS Exam Center in Velachery

    ReplyDelete
  50. Excellent Post! Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge.
    Embedded System Training in Chennai | Embedded Training in Velachery | Embedded Courses in Pallikaranai

    ReplyDelete
  51. This is useful post for me. I learn lot of new information from your article. keep sharing. thank you for share us.
    MCSE Training Institute in Chennai | MCSE Training in Velachery | MCSE Training Center in Chrompet

    ReplyDelete
  52. Thanks for giving nice information from your blog...It's really an amazing post..
    Selenium Training Institute in Chennai | Selenium Training Center in Velachery

    ReplyDelete
  53. It is amazing blog and good information... I was improve my knowledge... Thanks for sharing such a informative and wonderful post...
    Java Training Institute in Chennai | Java Training Center in Velachery | Java Certification Training in Taramani

    ReplyDelete
  54. Thanks for your informative article. Your post helped me to understand the future and career prospects. Keep on updating your blog with such awesome article.
    PCB Designing Training Institute in Chennai | PCB Training in Velachery

    ReplyDelete
  55. Wow...What an excellent informative blog, really helpful. Thank you so much for sharing such a wonderful article with us.keep updating..
    Hadoop Training Institute in Chennai | Hadoop Training in Velachery | Hadoop Training Center in Chennai

    ReplyDelete
  56. I am reading your post from the beginning, it was so interesting to read & I feel thanks to you for posting such a good blog, keep updates regularly.
    Embedded Training Institute in Chennai | Embedded Training in Velachery

    ReplyDelete
  57. This is really too useful and have more ideas from yours. keep sharing many techniques and thanks for sharing the amazing article.
    MatLab Training Institute in Chennai | MatLab Training Center in Velachery

    ReplyDelete
  58. Really nice post. Thank you for sharing your amazing information and informative article,its really useful for us.keep updating such a wonderful blog..
    Embedded Training Institute in Chennai | Embedded Training Center in Velachery

    ReplyDelete
  59. Very informative and interesting blog, it was so good to read and useful to improve my knowledge as updated one,keep updating..This Concepts is very nice Thanks for sharing..
    Selenium Training Institute in Chennai | Selenium Training Center in Velachery | Selenium Courses in T.Nagar

    ReplyDelete
  60. Thanks for sharing your great information..Its really very impressive and informative content.keep updating...
    Linux Certification Training Institute in Chennai | Linux Training in Velachery | Online Linux Training in Madipakkam

    ReplyDelete
  61. Amazing blog. Thank you for sharing. The information you shared is very effective for learners I have got some important suggestions from it..
    Blue Prism Training Institute in Chennai | Blue prism Certification Training in Velachery | Blue Prism Training Center in Adyar

    ReplyDelete
  62. Pretty article! I found some useful information in your blog, it was amazing to read, thanks for sharing this great content to my vision...
    Embedded Training Institute in Chennai | Embedded Training in Velachery | Embedded Certification Training in Velachery

    ReplyDelete
  63. Thank you so much for posting your amazing article...I really appreciate your work. Keep it up. Great work!...keep updating...
    Cloud Computing Training Institute in Chennai | Cloud Computing Training in Velachery

    ReplyDelete
  64. Excellent blog. Thank you for sharing. The information you shared is very effective for learners I have got some important suggestions from it..
    MatLab Training Institute in Chennai | MatLab Training in Velachery | MatLab Training in Taramani

    ReplyDelete
  65. Thanks for sharing this great article! That is very interesting I love reading and I am always searching for informative articles like this..
    Cisco Certification Training in Chennai | Cisco Certification Courses in OMR | Cisco Certification Exams in Velachery

    ReplyDelete
  66. Wow!!..What an excellent informative post, its really useful.Thank you so much for sharing such a awesome article with us.keep updating..
    VMware Certification Training in Chennai | VMware Training Institute in Velachery | VMware Certification Courses in Medavakkam

    ReplyDelete
  67. Amazing blog. Thank you for sharing. The information you shared is very effective for learners I have got some important suggestions from it..
    Embedded System Training Institute in Chennai | Embedded Training in Velachery | Embedded Courses in T.nagar

    ReplyDelete
  68. Great post.Thanks for one marvelous posting! I enjoyed reading it;The information was very useful.Keep the good work going on!!
    Tally Training Institute in Chennai | Tally Training in Velachery | Best Tally Courses in Guindy | Tally Training Center in Pallikaranai

    ReplyDelete
  69. I am reading your post from the beginning,it was so interesting to read & I feel thanks to you for posting such a good blog,keep updates regularly..
    Web Designing and Development Training in Chennai | Web Designing Training Center in Velachery | Web Design Courses in Pallikaranai

    ReplyDelete
  70. Awesome post.. Really you are done a wonderful job.thank for sharing such a wonderful information with us..please keep on updating..
    PCB Designing Training Institute in Chennai | PCB Training Center in Velachery | PCB Design Courses in Thiruvanmiyur

    ReplyDelete
  71. Thanks for making me this Blog. You have done a great job by sharing this content here.Keep writing blog this like.
    MatLab Training Institute in Chennai | MatLab Training Center in Velachery | MatLab Courses in Tambaram

    ReplyDelete
  72. Your article is really an wonderful with useful content, thank you so much for sharing such an informative information. keep updating.
    MultiMedia Training Center in Chennai | MultiMedia Training Courses in Velachery | MultiMedia Training Institutes in OMR

    ReplyDelete
  73. Pretty blog, so many ideas in a single site, thanks for the informative article, keep updating more article.
    Software Testing Training Institute in Chennai | Software Testing Training Institutes in Velachery

    ReplyDelete
  74. I am really enjoying reading your well-written articles. It looks like you spend a lot of effort and time on your blog..
    MCSE Certification Training Institute in Chennai | MCSE Training Center in Velachery

    ReplyDelete
  75. Wonderful article..This is very informative blog. Glad to found your blog.Helps to gain knowledge about new concepts and techniques. Thanks for posting information in this blog..
    Linux Certification Training Institute in Chennai | Linux Training Center in Velachery | Linux Courses in Medavakkam

    ReplyDelete
  76. This is useful post for me. I learn lot of new information from your post. keep sharing. thank you for share us..
    Web Designing Training Institute in Chennai | Web Design Taining Center in Velachery | Web Designing Courses in Taramani

    ReplyDelete
  77. Your blog is really useful for me, and I gathered some information from this blog.Thanks a lot for sharing this amazing article..
    CCNP Training Institute in Chennai | CCNP Training Center in Velachery | CCNP Training Courses in Pallikaranai | CCNP Training in Taramani | CCNP Courses in Medavakkam

    ReplyDelete
  78. Thanks for sharing this great article! That is very interesting I love reading and I am always searching for informative articles like this.
    MatLab Training Institute in Chennai | MatLab Training Center in Velachery | Matlab Courses in Adyar

    ReplyDelete